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PREFACE 


The  main  body  of  this  technical  report  is  an  article  prepared  for 
a special  issue  of  Communications  in  Statistics,  Part  A on  statistics  in 
weather  modification.  It  will  be  preprinted  also  in  the  Proceedings  of 
the  Workshop  on  Statistical  Design  and  Analysis  of  Weather  Modification 
Experiments,  Tallahassee,  October,  1978,  with  permission  of  the  Editor. 
Inclusion  of  the  article  as  part  of  this  technical  report  maintains  con- 
tinuity of  reporting  under  the  contract. 

The  article  is  based  partly  on  our  ONR  Technical  Report  No.  133,  FSU 
Statistics  Report  No.  M467.  Attention  is  directed  also  to  our  January  31, 
1979  Errata  to  that  technical  report,  with  those  corrections  included  in 
the  article.  Section  5 of  the  article  refers  to  research  by  Elton  Scott 
and  details  were  given  in  ONR  Technical  Report  No.  127,  FSU  Statistics 
Report  No.  M442.  A follow-on  report  by  Scott  is  in  preparation.  Sections 
4 and  6 of  the  article  contain  some  new  analyses  with  transformed  data  and 
some  comments  on  other  analyses  not  shown.  A Supplementary  Appendix,  at 
the  end  of  this  report,  shows  results  of  these  additional  analyses  not  re- 
ported previously. 


Ralph  A.  Bradley,  Sushi 1 S.  Srivastava  and  Adolf  Lanzdorf 

Florida  State  University 
Tallahassee,  Florida 


ABSTRACT 

Data  from  a weather  modification  experiment  are  examined  and 
a number  of  statistical  analyses  reported.  The  validity  of  ear- 
lier inferences  is  studied  as  are  the  utilities  of  various  sta- 
tistical methods.  The  experiment  is  described.  The  original 
analysis  of  North  American  Weather  Consultants,  who  conducted  the 
experiment,  is  reviewed.  Data  summarization  is  reported.  A major 
approach  to  analysis  is  through  the  use  of  cloud-physics  covari- 
ates in  regression  analyses.  Finally,  a multivariate  analysis  is 
discussed.  It  appears  that  the  covariates  may  have  been  affected 
by  treatment  (cloud  seeding)  and  that  their  use  is  invalid,  not 
only  reducing  error  variances  but  removing  treatment  effect.  Some 
recommendations  for  improved  design  of  similar  future  experiments 
are  given  in  a concluding  section,  including  preliminary  trial 
use  of  blocking  by  storms. 
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1.  INTRODUCTION 

Phase  I cf  the  Santa  Barbara  Convective  Seeding  Test  Program 
was  conducted  by  North  American  Weather  Consultants  (NAWC)  from 
1967  through  1971  for  the  Research  Department,  Naval  Weapons  Cen- 
ter, China  Lake,  California.  A concurrent  study  on  the  large- 

scale  effects  of  cloud  seeding  was  undertaken  for  the  Bureau  of 

* 

Reclamation,  U.S.  Department  of  the  Interior,  Denver,  Colorado  by 
Aerometric  Research  Inc.,  the  research  affiliate  of  NAWC.  Re- 
search on  the  design  and  analysis  of  weather  modification  experi- 
ments at  the  Florida  State  University  is  sponsored  by  the  Office 
of  Naval  Research,  Department  of  the  Navy,  Washington,  D.C.  Data 
from  the  Phase  I Santa  Barbara  experiment,  provided  through  the 
courtesy  of  NAWC,  have  been  used  for  trial  analyses.  Some  ap- 
proaches to  statistical  analysis  of  the  Phase  I data  are  reported 
in  this  article. 

A brief  summary  of  the  Phase  I experiment  follows.  More  de- 
tail is  given  in  technical  reports  by  Elliott  and  Thompson  (196Sa, 
1968b,  1969,  1972)  and  in  publications  by  Elliott,  St.  Amand,  and 
Thompson  (1971)  and  Brown,  Elliott,  Thompson,  St.  Amand  and  Elliott 
(1974) . Two  final  reports  were  issued,  one  for  the  Naval  Weapons 
Center  and  one  for  the  Bureau  of  Reclamation,  by  Thompson,  Brown 
and  Elliott  (1975)  and  Brown,  Thompson  and  Elliott  (1975)  respec- 
tively. Both  final  reports  include  Phase  II  experiment  results, 
1971  through  1974.  In  this  article,  attention  is  on  Phase  I data 
because  of  experimental  design  changes  and  the  introduction  of 
aerial  seeding  in  Phase  II.  Data  collected  for  the  Bureau  of 
Reclamation  study  is  used  unless  otherwise  specified  because  of 
its  augmented  network  of  raingages. 

Winter  storms  in  the  Santa  Barbara  region  have  identifiable 
convective  cells  grouped  into  bands,  usually  taking  from  one-half 
to  one  and  one-half  hours  to  pass  over  a point.  Several  convec- 
tive bands  may  occur  in  a storm  or  it  may  be  a single  froatal 
band.  The  convective  band  was  used  as  the  experimental  unit  in 
the  Phase  I experiment.  Criteria  for  the  "seedability"  of  a 
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convective  band  were  established;  in  operation,  they  reduced  to  a 
wind  flow  requirement  such  that  the  possible  effects  of  seeding 
would  fall  mainly  in  a target  area  and  the  expectation  of  substan- 
tail  precipitation.  Band  detection  was  either  by  radar,  confirmed 
through  precipitation  at  a telemetered  raingage  in  a control  area 
to  the  west  of  a single  ground  seeding  site,  or  through  precipi- 
tation at  two  such  telemetered  raingages-,  A predetermined  ran- 
domized decision  to  seed  or  not  seed  an  experimental  unit,  a 
seedable  convective  band,  was  applied.  Figure  1,  taken  from 
Elliott,  St.  Amand  and  Thompson  (1971),  depicts  the  general  ex- 
perimental set  up;  not  all  raingages  used  are  shown  nor  were  all 
raingages  always  in  operation.  Seeding  in  Phase  I was  ground 
based  from  a mountain  crest  at  1065m.  above  sea  level  indicated 
in  Figure  1.  High  output,  silver-iodide,  pyrotechnic  devices, 
ignited  at  15  minute  intervals  from  the  beginning  of  band  passage 
for  seeded  bands,  were  used. 

Band  precipitation  data  were  obtained  for  all  raingages  in 
control  and  target  areas  operable  for  a band.  (The  number  of 
raingages  was  increased  from  time  to  time  during  the  Phase  I ex- 
perimentation.) The  procedure  included: 

(i)  tracking  of  the  precipitation  band  pattern  across  the 
gage  network  on  the  basis  of  plots  of  available  precipitation  and 
radar  information, 

(ii)  determination  of  the  time  of  band  passage  (and  hence 
time  of  band  duration)  at  each  raingage,  and 

(iii)  calculation  of  total  precipitation  from  the  raingage 
record  attributable  to  the  band. 

To  avoid  subconscious  bias,  the  meteorological  analyst  determining 
raingage  band  passage  times  and  precipitations  was  uninformed  as 
to  which  bands  were  seeded.  Considerable  skill  was  required  from 
the  analyst.  A major  source  of  variation  may  arise  from  these 
determinations,  a disadvantage  in  the  use  of  convective  bands  as 
experimental  units,  perhaps  offset  by  the  resulting  increase  in 
the  number  of  experimental  units  available  in  a season. 


(1971).  Raingage  sites  are  designated  by  solid  or  open 
circles,  telemetered  gages  underlined.  The  seeding  and 
radar  site  is  indicated  by  a solid  triangle. 
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Air  mass  characteristics  of  each  band  were  determined  from 

radiosonde  observations  at  Santa  Barbara  Airport  and  Vandenberg 

Air  Force  Base  (VBG  in  Figure  1) • An  attempt  was  always  made  to 

obtain  a sounding  as  a band  passed  over  the  airport.  Gleeson 

(1977)  summarized  the  meteorological  data  with  a view  to  their  use 

as  covariates.  The  use  of  covariate  analysis  for  the  reduction  of 

experimental  error  and  improved  sensitivity  in  evaluations  of  the 

effects  of  seeding  seemed  a desirable  approach  to  evaluation  of 

weather  modification  experiments.  Gleeson  provided  data  for  each 

band  (except  Band  73)  on  the  following  variables: 

Xji  Mixing  Ratio, 

X.:  700  mb  Wind  Speed, 

X^:  700  mb  Wind  Direction, 

X4:  Mean  Wind  Speed, 

Xj.*  Direction,  Avg.  Vector  Wind, 

X,:  500  mb  Temperature,  ,, 

o U'l) 

X?:  Stability  Class, 

Xg:  Showalter  Index, 

X^:  Stability  Wind  Speed, 

Xjq:  Direction,  Stability  Wind, 

Instability  Transport, 

X12 : Band  Passage  Time  (Seeding  Site). 

More  detailed  descriptions  of  these  variables  are  given  by 
Gleeson. 

The  data  array  for  the  Phase  I experimentation  may  be  viewed 
as  a data  matrix  with  N rows  or  bands,  the  first  rows  for 

unseeded  bands  and  the  second  N2  rows  for  seeded  bands,  N * 107, 
Nj^  » 51,  N2  * 56,  and  with  columns  containing  precipitation  re- 
sponses at  individual  raingages,  possibly  grouped  by  locations, 
and  values  of  the  concomitant  variables,  Xj  to  Xj,.  The  data  are 
not  without  problems.  Raingage  precipitation  responses  arc  cor- 
related (correlation  approximately  0.6)  as  would  be  expected. 

There  are  missing  data  for  many  gages.  There  may  be  problems 
also  in  consideration  of  rows  as  independent  observation  vectors. 
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Elliott  and  Thompson  (1968b)  consider  the  persistence  effects  of 
seeding  and  conclude:  "some  seeding  precipitation  enhancement  may 
have  occurred  in  non-seeded  bands  which  follow  on  seeded  bands". 
NAWC  analyses,  Elliott  and  Thompson  (1969),  suggest  the  possibil- 
ity of  an  up-wind  effect  west  of  the  seeding  site  attributable  not 
to  westward  seeding  contamination  but  to  seeding-caused  blocking 
of  the  air-mass  flow  leading  to  up-wind  convection  development. 
Bradley,  Srivastava  and  Lanzdorf  (1977a)  provide  precipitation 
summarization  data  used  below.  These  data,  together  with  those 
of  Gleeson,  are  available  to  readers  interested  in  investigating 
other  approaches  to  the  analysis  of  this  weather  modification 
experiment . 

Primary  NAWC  analyses  are  reviewed  in  the  next  section.  This 
is  followed  by  a short  discussion  of  the  authors'  efforts  to  sum- 
marize the  precipitation  data.  The  use  of  the  available  data  in 
regression-covariance  analyses  is  reported,  followed  by  a pre- 
liminary multivariate  analysis.  The  article  concludes  with  some 
remarks  on  the  design  of  similar,  future  weather  modification  ex- 
periments . 

2.  NAWC  DATA  ANALYSIS 

The  main  NAWC  approach  to  data  analysis  was  on  a raingage 
station- by-station  basis. 

Let  y^Q  denote  precipitation  at  station  i from  band  ct, 

a * 1,...,N.  Let  Y0(i)  =1  or  0 as  station  i was  operative  or 

not  operative  for  band  a and  let  6 (i)  = 1 or  0 as  band  a was 

seeded  or  not  seeded.  Then  Yy  (i)  ® N(i)  and  To  (i)y  (i)  ■ N*  (i), 

a _ a a s 

a oi 

respectively  the  number  of  observations  and  the  number  of  seeded 
bands  recorded  at  station  i.  The  number  of  unseeded  bands  at 
station  i is  Nns(i.)  * N(i)  - Ng(i).  Then 

Ts(i)  - Iyi)Ya(i)yia/Ns(i)  (2.1) 

01 

and 

T„sCi)  * IU-*a(i)]T4(i)yla/nn,(i)  r-2) 

a 
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are  precipitation  averages  at  station  i for  seeded  and  non-seeded 
bands  respectively.  Six  control  area  detection  stations  were  used 
stations  circled  in  Figure  1.  If  k indexes  these  control  sta- 
tions, define 

= IT  m/6  (2.3) 

S k S 
and 


ZT„  (k)/6'. 


ns  “ ns 
k 

The  descriptive  statistic  used  by  NAWC  for  station  i,  a double 
ratio,  is 


(2.4) 


DR(i)  - [Ts(i)/Cs]/[Tns(i)/Cns].  (2.S) 

Use  of  the  double  ratio  was  compared  with  use  of  the  single  ratio, 

SR(i)  = Ts(i)/Tns(i).  (2.6) 

It  was  found  that  much  the  same  results  were  obtained  for  the  two 
statistics.  The  intent  in  use  of  (2.5)  was  to  standardize  the 
comparison  of  seeded  and  non-seeded  responses  through  divisions  by 
control  area  precipitations,  assumed  to  be  unaffected  by  seeding. 
The  control  area  detection  stations  had  only  a few  missing  obser- 
vations and  apparently  Cg  and  Cns  in  (2.3)  and  (2.4)  were  then 
evaluated  from  the  available  observations. 

Figure  2,  based  on  the  Naval  Weapons  Center  study,  shows  con- 
tours for  the  double  ratio  of  (2.5)  for  the  Phase  I experiment. 
Similar  figures  are  given  by  Elliott  and  Thompson  (1972)  for  the 
single  ratios  of  (2.6'  and  for  subdivisions  of  the  data  by  years, 
stability  classes,  and  500  mb  temperatures.  The  locations  of  re- 
gions of  possible  precipitation  enhancement  are  fairly  stable  in 
all  such  figures  and  they  tend  to  be  regions  with  low  average  pre- 
cipitations for  both  seeded  and  unseeded  bands. 

NAWC  states  in  their  various  reports  that  the  Wilcoxon-Mann- 
Whitney,  two-sample,  rank  test  was  used  to  assess  the  significance 
of  double  and  single  ratios  for  each  raingage  station.  Signify 
canoes  were  noted-  os--in  Figure-2.  The  methods  of  application 
of  the  test  are  not  clear  in  the  reports  but  we  give  our  under- 
standings. For  the  single  ratio,  y A was  used;  the  precipitation 
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measurements  themselves  were  grouped  into  two  samples,  seeded  and 

unseeded,  and  the  rank  test  applied.  For  the  double  ratio,  y.  /C 

_ 1 ia  a 

was  calculated  for  each  band  a at  stations  i,  Cq  being  the  aver- 
age of  the  six  (or  available)  control-area,  detection  stations  for 
band  o,  and  these  indices  were  grouped  into  two  samples  as  before 
and  the  test  applied  for  station  i. 

Similar  analyses  were  completed  by - NAWC  using  band  duration 
times  instead  of  precipitations  as  the  responses  and  very  similar 
results  were  obtained.  See  Figure  5.12  of  Elliott  and  Thompson 
(1972) . We  concentrate  on  precipitation  analyses  in  this  article, 
but  the  possibility  that  seeding  may  affect  duration  time  is  dis- 
cussed below. 

NAWC  was  aware  that  their  analyses  were  open  to  possible  cri- 
ticisms. Possible  persistence  effects  of  seeding  would  seem  to 
decrease  the  apparent  effects  of  seeding  and  raise  questions  about 
the  use  of  convective  bands  as  independent  experimental  units. 
Station-by-station  tests  of  significance  are  not  independent.  The 
responses  y^Q  for  the  various  stations  i have  correlations  of 
approximately  0.6  as  noted  above.  The  indices  y^  /(T  have  ad- 
ditional dependencies  because  the  denominators  are  the  same  for 
all  stations  for  band  o.  Concerned  with  these  dependencies,  NAWC 
conducted  a limited  Monte  Carlo  study,  reported  by  Elliott  and 
Brown  (1971)  in  their  Table  1,  to  give  additional  credence  to 
their  concludions.  They  state:  "At  the  0.05  significance  level 
for  all  bands,  29  stations  in  the  original  test  sample  were  found 
to  show  a positive  difference  between  seeded  and  not-seeded  cases; 
and  three  Monte  Carlo  runs  (out  of  50)  were  found  to  have  as  high 
or  higher  counts  of  stations  with  a positive  difference  at  this 
significance  level."  The  use  of  ratios  to  measure  precipitation 
enhancement  is  open  to  question  depending  on  project  objectives. 

If  large  ratios  occur  in  areas  of  relatively  low  precipitation, 
somewhat  sparsely  represented  by  raingages,  the  effect  on  total 
or  average  precipitation  for  a larger  defined  target  area  may  be 
small  and  the  result  of  little  economic  value. 


3.  DATA  SUMMARIZATION 


A more  direct  approach  to  the  analysis  of  a weather  modifi- 
cation experiment  is  to  consider  summary  measures  of  precipitation 
for  each  experimental  unit  over  designated  response  areas.  The 
arithmetic  mean  of  the  raingage  measurements  over  a response  area 
for  each  unit  would  be  the  summary  measure  typically  used. 

Bradley,  Srivastava  and  Lanzdorf  (19 77a, b)  defined  response 
areas  as  in  Table  I.  The  locations  of  these  areas  may  be  identi- 
fied through  reference  to  Figure  2.  The  first  five 


TABLE  I 

Definitions  of  Response  Areas 


Response  Ranges  in  Degrees  Number  of 

Area  Latitude  Longitude  Stations 


(i)  34.0-35. 25  118.0-120.02  107 

(ii)  34.4-35.0  119.51-120.02  26 

(iii)  34.0-35.0  118.0-119.51  72 

(iv)  Areas  (ii)  + (iii)  98 

(v)  All  Stations  in  the  Naval  Weapons  Center  61 
Reports  East  of  Seeding  Site,  long.  120.02 

Control* *  34.4-35.25  120.02-120.60  34 


*The  Control  Area  for  the  Naval  Weapons  Center  study  con- 
sists of  all  39  stations  west  of  the  seeding  site. 


areas  will  be  referenced  as  Target  Areas  and  the  last  as  the  Con- 
trol Area.  The  number  of  raingage  stations  and  the  data  used  for 
Target  Areas  (i)-(iv)  are  those  of  the  Bureau  of  Reclamation  study 
and  those  for  Target  Area  (v)  are  those  of  the  Naval  Weapons  Center 
study  with  minor  modifications  noted  in  the  two  cited  references. 
Note  that  these  target  areas  cover  the  test  area  of  Figure  1,  but 
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that  some  raingages  existed  outside  of  these  response  areas, 
some  of  them  in  arid  regions. 

Precipitation  averages  in  inches  are  exhibited  in  Table  II 
for  the  various  response  areas.  They  were  computed  as  simple 
averages  of  the  individual  convective  band  averages  of  available 
raingage  measurements  for  the  band  in  the  designated  response 

area.  The  numbers  of  raingages  available  increased 

* 

« 

TABLE  II 

Precipitation  Means  in  Inches 


Response  Areas 

(i) 

(ii) 

(iii) 

(iv) 

(v) 

Control 

Seeded  Bands 
Uhseeded  Bands 

0.2S7 

0.178 

0.329 

0.229 

0.249 

0.172 

0.271 

0.187 

0.267 

0.190 

0.234 

0.203 

somewhat  with  the  seasons  and  not  all  raingages  were  operable  for 
all  convective  bands.  Table  II  is  intended  only  to  indicate  the 
nature  of  responses.  It  reinforces  impressions  given  by  Figure  2 
with  its  double  ratios.  The  Control  Area  mean  for  seeded  bands 
is  higher  than  that  for  unseeded  bands,  as  are  Target  Area  means, 
suggesting  either  that  seeding  had  some  effect  in  the  Control 
Area  or  misfortune  in  the  randomized  choices  of  bands  to  be 
seeded. 

With  the  intent  of  improvement  of  data  summarization,  Bradley, 
Srivastava  and  Lanzdorf  (1977a, b)  summarized  the  precipitation 
data  through  the  use  of  response  surfaces  for  the  Control  Area  and 
Target  Area  (i)  separately.  The  basic  independent  variables  were 
the  coordinates  of  latitude  and  longitude  for  the  raingage  sta- 
tions with  individual,  raingage  precipitation  measurements  as  the 
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dependent  variable  observations.  Separate  response  surfaces  were 

found  for  each  convective  band.  It  was  found  necessary  to  use 

general  cubic  response  models  to  represent  responses  adequately. 

Precipitation  volumes  and  their  variances  were  calculated,  the 

volumes  obtained  through  integrations  of  the  surfaces  over  the 

« 

designated  target  areas  or  control  area..  Figure  5 is  typical  of 
results  obtained;  the  region  where  the  surface  is  negative  is  off- 
shore. 

The  response  surface  approach  was  successful  as  a method  of 
data  summarization.  It  was  not  successful  in  improvement  of  data 
summarization  in  comparison  with  use  of  the  raingage  means  over 
stations  within  response  areas  for  a convective  band.  Some  70% 
of  the  inherent  variation  in  responses  among  raingages  within  a 
band  and  response  area  was  explained  by  the  independent  variables, 
the  percentages  varying  considerably  from  band  to  band.  Residuals 
from  fitted  surfaces  exhibited  only  limited  spatial  trends  when 
the  cubic  surfaces  were  used.  Correlations  between  precipitation 
volumes  calculated  from  the  response  surfaces  and  precipitation 
means  were  given  by  Bradley,  Srivastava  and  Lanzdorf  (1977a, b) . 
They  ranged  from  0.97  to  0.99  for  Target  Areas  (i)-(iv)  and  the 
correlation  was  0.89  for  the  Control  Area.  The  use  of  volumes 
in  consideration  of  the  effects  of  seeding  cannot  be  expected  to 
yield  additional  insights,  although  Bradley,  Srivastava  and  Lanz- 
dorf (1978)  did  examine  their  use  as  reported  in  the  next  section. 

Scott  (1978)  used  a multivariate  approach  to  data  summari- 
zation. He  found  principal  components  among  raingage  responses 
in  both  Target  Area  (i)  and  the  Control  Area  with  a view  to  sum- 
marizing responses  through  one  or  more  orthogonal  linear  combi- 
nations of  the  raingage  measurements  of  stations  in  response  areas 
for  each  band.  Thus,  raingage  responses  were  treated  as  variates 
and  convective  bands  as  experimental  units.  Substantial  pruning 
of  the  data  and  some  innovations  were  required  to  circumvent  the 
serious  problems  of  missing  observations  in  multivariate  analysis. 
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Correlation  matrices  were  obtained  and  principal  components  de- 
termined, along  with  their  eigenvalues.  The  first  three  principal 
components  were  interpretable  approximately  as  a mean  response,  a 
coastal  versus  inland  contrast,  and  an  East-West  contrast.  Per- 
centages of  variation  explained  by  these  components  were  respec- 
tively 71.3,  6.7  and  5.9  in  Target  Area  (i)  and  76.1,  6.7  and  4.7 
in  the  Control  Area.  The  correlations  of  the  first  component  with 
the  band  mean  were  0.997  for  Target  Area  (i)  and  0.985  for  the 
Control  Area.  Scott  is  engaged  in  use  of  these  results  in  examin- 
ation of  the  effects  of  seeding.  The  first  component  cannot  be 
expected  to  yield  new  insights;  other  components  may  add  some  new 
information. 

4.  COVARIANCE  ANALYSES 

Weather  modification  experiments  are  conducted  necessarily 
in  a natural  environment  involving  much  variability.  The  use  of 
covariates  in  analyses  for  the  reduction  of  experimental  error 
appears  to  be  the  major  available  means  to  improved  experimental 
design.  It  was  for  this  purpose  that  Gleeson  (1977)  summarized 
information  on  covariates  as  discussed  in  Section  1.  We  report 
in  this  section  on  covariance  analyses  effected  through  use  of 
multiple  regression  methods. 

Bradley,  Srivastava  and  Lanzdorf  (1978)  reported  on  initial 
covariance  analyses.  (Some  later  analyses  are  reported  below.) 
Regression  models  used  were  of  the  form, 

U =•  3n  ♦ & V.  ♦ 6Z  + e,  (4.1) 

u i-1  1 1 

where  U is  a precipitation  response  variable  for  a target  area, 

is  the  i1*1  covariate,  Z = 1 or  0 as  the  experimental  unit  was 

or  was  not  seeded,  the  0's  and  6 are  regression  parameters, 

and  c is  a random  error.  The  data  matrix  has  rows, 

(U  , V,  , . . . , V , Z ) , a a I N.  The  regression  parameters 
v a la  pa  a 

were  estimated  by  weighted  least  squares  through  minimization  of 
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In  the  referenced  report,  use  of  the  set  of  covariatcs  of  (L.l) 
and  their  interactions  with  treatment  (seeding),  along  with  Xc, 
a measure  of  Control  Area  precipitation,  was  explored.  Both  tar- 
get area  mean  precipitation  and  target  ajrea  precipitation  volume, 
see  Section  3,  were  used  as  U for  Target  Areas  (i)-(iv),  together 
with  corresponding  measures  for  X . No  results  are  summarized 
here  for  precipitation  volumes  since  they  were  very  similar  to 
those  for  mean  precipitations.  Pairwise  unweighted  correlation 
coefficients  are  shown  in  Table  A-I  for  mean  precipitation,  X£, 
and  the  covariates  of  (1.1)  to  give  an  indication  of  relationships 
for  Target  Area  (i).  Note  that  X£,  Control  Area  mean  precipi- 
tation, and  Xj2»  Band  Passage  Time,  correlate  most  highly  with 
target  area  mean  precipitation;  both  of  these  covariates  may  be 
affected  by  seeding  — we  have  noted  a possible  effect  of  seeding 
up-wind  from  the  seeding  site  in  the  Control  Area  and  it  has  been 
conjectured  that  the  effect  of  seeding  may  be  to  increase  rainfall 
through  an  increase  in  band  passage  time. 

It  was  reported  in  the  reference,  after  preliminary  analyses, 
that  seven  of  the  twelve  covariates  of  (1.1)  were  sufficient  for 
experimental  error  reduction.  They  were: 


V 

V 


700  mb  Wind  Speed, 

700  mb  Wind  Direction, 
500  mb  Temperature, 
Stability  Class, 
Showalter  Index, 

: Instability  Transport , 


(4.3) 


8‘ 

X11 

X12:  duration  of  Band  Passage. 


The  selection  was  based  on  redundancy  considerations  and  their 
contributions  to  error  reduction.  Final  analyses  were  done  for 
four  models  with  unit  weights  (unweighted)  and  weights. 


w * n /s2,  where 
a a a 


n^  is  the  number  of  raingage  observations 
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contributing  to  the  precipitation  mean  for  band  a and  s2  is 


the  variance  among  those  observations.  The  models  in  the  form 


(4.1)  had  the  covariates  as  follows: 


Model 


Identification  of  V.  ,...,V 

11 -JL 


(1) 

(2) 


X2'  X3’  X6’  X7’  X8*  Xll’  X12‘ 


C3) 

(4) 


V's  of  Model  (1)  plus  X-Z,  XI, 
X6Z*  X7Z’  X8Z'  X11Z'  X12Z" 

X£  plus  V’s  of  Model  (1). 


(4.4) 


Xc  plus  V's  of  Model  (2) . 


Values  of  the  coefficient  of  determination  R2,  the  square  of  the 
multiple  correlation  coefficient  for  N = 106  bands*,  are  given 
for  the  four  models  and  Target  Areas  (i)-(v)  in  Table  III. 


TABLE  III 


Coefficients  of  Determination  (R2)  for 
Regressions  with  Precipitation  Means 


Models  Without  Control  Mean 
Target  Area 

Unweighted  Weighted** 


(1) 

(2) 

U) 

(2) 

(3) 

(4) 

(3) 

(4) 

(i) 

0.597 

0.621 

0.364 

0.437 

0.712 

0.750 

0.593 

0.615 

(ii) 

0.578 

0.608 

0.373 

0.457 

0.789 

0.815 

0.681 

0.691 

(iii) 

0.578 

0.606 

0.285 

0.468 

0.646 

0.691 

0.505 

0.582 

(iv) 

0.604 

0.629 

0.371 

0.442 

0.712 

0.751 

0.589 

0.610 

(v) 

0.575 

0.593 

0.344 

0.426 

0.778 

0.805 

0.603 

0.627 

Models  With  Control  Mean 


Unweighted  Weighted* 


**See  Table  A-5,  Bradley,  Srivastava  and  Lanzdorf  (197S). 
Values  of  R2  in  the  reference  for  weighted  regressions  have 
been  corrected. 


*See  Clccson  (1977);  there  were  107  bands  but  covariatc  data 

were  missing  for  Band  73. 


k 
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Bradley,  Srivastava  and  Lanzdorf  (1978)  gave  estimates  of  the  re- 
gression parameters  and  corresponding  analysis  of  variance  tables 
with  sources  of  variation  being  reduction  in  variation  due  to 
basic  covariates,  additional  reductions  due  to  interactions  (when 
included  in  the  model)  of  the  basic  covariates  with  seeding,  final 
reduction  due  to  seeding,  and  residual  variation.  Results  were 
disappointing.  There  were  no  apparent  effects  due  to  seeding. 
There  was  little  interaction.  The  combined  effects  of  the  basic 
covariates  were  significant,  generally  at  the  0.01  level  of  sig- 
nificance. 

We  were  not  satisfied  with  the  preliminary  analyses.  Stan- 
dard deviations  were  related  to  means  as  seen  in  Figure  4 below 
for  Target  Area  (i).  Values  of  n^  varied  also.  The  weighted 

analyses  gave  very  heavy  weights  to  bands  with  low  precipitation 
2 

means;  values  of  R were  reduced  as  seen  in  Table  III  and  weighted 
means  were  quite  different  from  the  unweighted  means  of  Table  II, 
often  suggesting  more  precipitation  for  unseeded  bands.  We  re- 
port now  on  new  analyses  with  the  data  transformed  to  stabilize 
variances . 

Analysis  of  the  data  of  Figure  4 and  similar  data  for  the 
other  target  areas  suggested  the  use  of  logarithmic  transforma- 
tions to  stabilize  variances.  Given  a raingage  observation  y, 
the  transformed  responses  were  of  the  form,  log  (1+ay).  Correla- 
tions with  Uj,  the  target  area  mean  of  the  transformed  responses, 
are  shown  in  Table  A- 1 for  Target  Area  (i);  they  are  very  close 
to  those  for  , the  target  area  mean  precipitation.  Figure  S 
shows  the  standard  deviations  of  the  transformed  responses  plotted 
against  values  of  U2  for  Target  Area  (i).  It  is  seen  that 
variances  have  been  stabilized  except  for  small  values  of  U^, 
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values  for  convective  bands  that  may  not  have  been  acceptable 
"seedabie"  bands. 

Regression  analyses  similar  to  those  described  above  were 
done.  Models  with  and  without  X and  X.-  were  used  because 

C 14 

it  had  been  suggested  that  they  may  have  been  affected  by  seeding. 
The  response  variable  for  each  model  is  the  mean  of  the  trans- 
formed precipitations  noted  above  for  tlte  designated  target 
area  and  band.  The  weights  wq  in  (4.1)  were  taken  to  be  n^, 
the  number  of  raingages  operative  in  the  target  area  for  band  o. 
The  models  used  were  as  follows: 

Model Identification  of  V^,...,Vp 

(5)  Xc,  X2,  X3,  X6,  X?,  Xg,  Xn,  X12. 

(6)  Model  (S)  less  Xc> 

(7)  Model  (5)  less  Xir  (4.5) 

(8)  Model  (5)  less  X£,  X12. 

(9)  Model  (5)  plus  X.Z,  X.Z,  X,Z,  X7Z, 

x8z,  xnz,  x12z.2  •> 

Results  from  these  newer  regression  analyses  are  summarised. 
Table  IV  shows  values  of  R2  that  may  be  compared  with  those  of 
Table  III.  In  particular,  values  for  models  (5)  and  (6)  of  Table 
IV  may  be  compared  respectively  with  those  for  models  (3)  and  (1) 

TABLE  IV 

Coefficients  of  Determination  (R2)  for  Regressions 
with  Means  of  Transformed  Precipitations 


Target 

Models 

Area 

(5) 

(6) 

(7) 

(8) 

(9> 

.293 

0.74S 

.236 

0.788 

.299 

0.696 

.302 

0.752 

: 

.229 

0.764 
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of  Table  III.  Slightly  larger  values  of  R2  were  obtained  with 

the  transformed  data.  Results  for  model  (8)  show  that  R2  is 

reduced  considerably  when  X and  X.,  are  omitted  as  covariates. 

2 c 12 

Model  (9)  has  values  of  R similar  to  but  slightly  larger  than 
for  Model  (5).  Table  A- II  contains  the  essentials  of 
analysis  of  variance  tables  for  models  (5) -(9)  for  the  five  target 
areas  of  Table  I based  on  the  transformed  data.  It  is  seen  that 
use  of  X^»  Duration  of  Band  Passage  and  X^  Control  Area  mean 
precipitation,  to  a lesser  extent,  as  covariates  reduces  the  ap- 
parent effect  of  seeding;  F-ratios  for  Seeding  are  largest  for 
model  (8)  without  inclusion  of  either  Xc  or  Xp.  Regression 
coefficients  for  model  (5)  are  given  in  Table  A- III  for  all  five 
target  areas.  These  values  permit  reconstruction  of  the  estimated 
regression  models  and  reinforce  comments  relative  to  X£  and  Xp 
above.  Examination  of  residuals  about  estimated  regression  models 
for  the  transformed  data  suggests  that  transformation  improved 
symmetry  and  approximate  normality  of  their  distributions. 

Gleeson  (1977)  saw  no  major  differences  between  results  on 
covariates  for  seeded  and  unseeded  bands  but  he  did  observe  that 
the  differences  exhibited  some  consistency.  After  some  discussion, 
he  wrote:  "Taken  separately  the  effect  of  these  differences  may 
be  insignificant,  but  in  combination  they  suggest  that  the  total 
precipitation  that  might  have  been  realized  from  seeded  bands,  had 
they  not  been  seeded,  would  have  been  larger  than  the  total  amount 
that  fell  from  nonseeded  bands."  Gleeson’ s concern  could  be  ex- 
plained by  unfortunate  randomization  in  the  seeding  decision  or 
through  a seeding  effect  on  the  covariates.  We  are  inclined  to 
the  latter  possibility.  We  have  referred  to  Elliott  and  Thompson 
(1969),  who  raised  the  possibility  of  an  up-wind  effect  in  the 
Control  Area  that  would  affect  X£.  Brown  and  Elliott  (1972),  in 
discussing  the  time  duration  of  a seeded  band,  state:  "There  is 
some  evidence  that  this  increased  duration  is  caused  by  a slowing 
down  of  the  back  edge  of  the  band  as  it  moves  across  the  area  of 
effect."  This  could  affect  X|V  Other  covariates  were  measured 
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by  radiosonde  at  Santa  Barbara  Airport,  well  into  the  target  area, 

and  their  values  may  have  been  affected  by  seeding  also.  Because 

of  these  concerns,  we  have  done  analyses  of  the  transformed  data 

omitting  all  covariates  but  retaining  the  weights,  w = n . 

a a 

Table  V shows  results  for  analyses  of  variance  for  the  trans- 
formed data  without  use  of  covariates.  The  appropriate  test  should 

* 

be  one-sided  and,  in  each  case,  the  regression  coefficient  for 
seeding  was  positive.  Assessment  of  t = /F  leads  to  a one-sided 
significance  level  of  0.05  or  slightly  larger  for  each  target  area. 
While  the  tests  are  not  independent,  these  results  confirm  those 
of  the  Monte  Carlo  assessment  of  the  NAWC  analysis  of  Section  2. 


TABLE  V 

Analysis  of  Variance  without  Covariates  for 


the 

Various  Target 

Areas, 

Transformed 

Data 

Target 

Area 

Source  of 

Variation 

d.f . 

Mean 

Squares 

F- 

Ratio 

(i) 

Seeding 

1 

110.29 

2.77 

Residual 

104 

39.84 

- 

(ii) 

Seeding 

1 

38.57 

2.84 

Residual 

104 

13.59 

- 

(iii) 

Seeding 

1 

79.49 

2.52 

Residual 

104 

31.55 

- 

(iv) 

Seeding 

1 

100.33 

2.76 

Residual 

104 

36.30 

- 

(v) 

Seeding 

1 

73.01 

2.58 

Residual 

104 

28.32 

- 

We  have  now  highlighted  what  may  be  the  major  design  defect 
in  the  Phase  I Santa  Barbara  experiment,  perhaps  a defect  that 
could  not  have  been  anticipated.  The  result  has  been  that  the 
covariance  analyses  have  not  been  helpful,  and,  indeed,  represent 
a misuse  of  the  method,  one  that  commonly  occurs.  Nevertheless, 
we  believe  that  covariance  analysis  should  be  a good  means  to 
improved  experimental  precision.  In  future  experiments,  attention 
should  be  given  to  choice  of  good  covariates  measured  in  appro- 
priate locations,  free  from  the  effects  of  seeding.  Perhaps 
measurements  at  Vandenberg  Air  Force  Base,  well  west  of  the 
seeding  site,  prior  to  seeding,  would  have  been  suitable. 

The  analyses  of  this  section  are  open  to  minor  technical  con- 
cerns. The  persistence  effect  of  seeding  again  raises  questions 
about  the  independence  of  experimental  units.  Normality  assump- 
tions are  not  valid  for  individual  raingage  observations  but 
should  be  appropriate  for  target  area  means.  Variance  heteroge- 
neity is  present,  but  should  be  of  little  concern  for  analyses 

with  transformed  data.  Choice  of  weights,  w = n , for  analyses 

a a 

with  transformed  data  is  only  strictly  appropriate  if  raingage 
observations  are  independent.  Independent  variables  in  the  re- 
gression models  are  subject  to  experimental  errors.  While  these 
concerns  are  present,  we  do  not  believe  that  analyses  should  be 
misleading,  particularly  when  the  transformations  are  used. 

5.  A MULTIVARIATE  ANALYSIS 

Somewhat  in  the  spirit  of  the  NAWC  station-by-station  analy- 
sis, but  without  the  problem  of  correlated  univariate  tests,  we 
have  performed  a crude  multivariate  analysis  without  use  of  co- 
variates or  transformations.  We  raport  briefly  here  since  several 
problems  with  the  multivariate  approach  arise. 

We  could  not  treat  the  response  from  each  target  area  rain- 
gage as  a separate  response  since  the  number  of  raingage  stations 
exceeds  the  number  of  experimental  units  and  because  there  would 
be  many  missing  observations.  Target  Area  (i)  was  divided  as  a 
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3><4  grid  creating  12  subareas  as  defined  in  Table  VI.  In  that 
table  we  show  simple  means  y"  and  y for  50  seeded  and  41 
unseeded  bands  for  each  subarea,  together  with  n,  the  number  of 
raingages  operative  on  average  in  the  subarea.  Values  of  t are 
given  for  the  univariate,  two- sample  Student  test  for  subareas; 
one-sided  significance  levels  vary  from  0.025  to  0.149.  Only  91 
bands  could  be  used  because  16  bands  had1 no  operative  stations 
in  one  or  another  of  the  subareas. 


TABLE  VI 

Subareas  of  Target  Area  (i) , Seeded  and  Unseeded  Means,  and 
Values  of  the  Student  Statistic 


Latitudes 

1 

119.5°-120° 

Longitudes 

119°-119.5°  118.5°-119° 

118°-118.5° 

n 

2.8 

8.5 

16.4 

2 1.6 

• o 

\*ns 

.214 

.186 

.156 

.168 

34  -34.41 

ys 

.285 

.269 

.240 

.266 

t 

1.14 

1.45 

1 .SS 

1.74 

n 

18.4 

7.4 

4.8 

4.1 

y 

.228 

.182 

.145 

.080 

34.41°- 34. 82° 

7 ns 

.364 

.285 

.245 

.145 

t 

1.97 

1.80  • 

1.90 

1.74 

n 

6.2 

5.6 

1.5 

1.6 

y 

.054 

.038 

.042 

.057 

34.82°-35.25° 

7 ns 

*s 

.085 

.065 

.075 

.084 

t 

1.49 

1.54 

1.53 

1.05 

The  multivariate  approach  considers  the  mean  response  per 
band  for  each  subarea  as  one  of  12  variates  and  the  two-sample 
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Hotelling  test  is  applied  with  sample  sizes  of  50  and  41.  The 
F-statistic  associated  with  the  test  has  the  value  0.81  with  12 
and  78  degrees  of  freedom  and  a two-sided  significance  level  of 
0.64,  not  indicative  of  a seeding  effect,  and  less  indicative  of 
such  an  effect  than  any  of  the  subarea  Student  tests,  all  of  which 
have  consistently  positive  values  of  t; 

What  has  happened  in  the  multivariate  test?  The  sample  disr 
persion  matrix  leads  to  correlations  between  subarea  means  of  ap- 
proximately 0.8,  larger  for  subareas  close  together  and  smaller 
for  subareas  farther  apart.  In  the  computation  of  the  Hotelling 
statistic,  the  quadratic  form  involved  has  a matrix  in  which  all 
non-diagonal  terms  are  negative  and  they  are  associated  with  cross- 
products  of  the  variates  that  are  always  positive.  Thus  the  qua- 
dratic form,  when  evaluated,  has  a value  much  reduced  from  the 
sum  of  its  terms  involving  squared  variate  observations.  The 
high  positive  correlations  between  subarea  means  have  reduced 
greatly  the  effectiveness  of  the  multivariate  analysis. 

There  are  other  problems  with  the  multivariate  approach. 

The  numbers  of  raingage  stations  operative  in  subareas  vary  from 
band  to  band  and  hence  observation  vectors  are  not  identically 
distributed;  in  particular,  they  do  not  have  a common  dispersion 
matrix.  In  addition,  comparison  of  the  two  sample  dispersion  ma- 
trices for  seeded  and  unseeded  bands  shows  that  they  are  not  the 
same  at  a high  level  of  significance  and  the  multivariate  Behrens- 
Fisher  problem  arises.  As  in  all  other  analyses,  independence  of 
observation  vectors  is  suspect  because  of  possible  persistence 
effects  of  seeding. 

Multivariate  analysis  does  not  appear  to  be  a likely  way  to 
improved  future  experiment  design  and  analysis.  Missing  obser- 
vations cause  great  difficulty  and  lead  to  exclusion  of  experi- 
mental units  containing  good  information.  Only  the  most  rigorous 
effort  to  avoid  missing  data  could  obviate  t lie  difficulty.  The 
high  correlations  among  subarea  means  would  require  development 
of  special  methods  for  efficient  analysis  of  resulting  data. 
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6_. Additional  Analyses,  Remarks  and  Recommendations 

After  completion  of  the  analyses  reported  above  and  develop- 
ment of  a preliminary  manuscript  for  this  article,  the  reports 

of  the  Weather  Modification  Board  (1978)  and  of  its  Statistical 

« 

Task  Force  (1978)  became  available.  The  first  emphasizes  the 
national  importance  of  weather  modification  and  the  need  for  much 
future  research.  The  second  addresses  many  of  the  statistical 
problems  associated  with  such  research,  delineating  between  ex- 
ploratory and  confirmatory  experimentation.  Bril linger,  Jones 
and  Tukey,  in  the  second  report,  emphasize  the  need  for  good  co- 
variates unaffected  by  seeding,  blocking  of  experimental  units, 
and  the  need  for  randomization  analyses.  We  have  regarded  the 
Phase  I Santa  Barbara  experiment  as  exploratory  and  parametric 
analyses  as  an  appropriate  and  efficient  approach  to  exploration 
of  the  data  for  new  insights  into  improved  future  experimental 
design. 

An  opportunity  for  the  blocking  of  convective  bands  by 
storms  was  not  used  in  the  design  of  the  Phase  I Santa  Barbara 
experiment.  The  randomized  decision  on  seeding  was  made  for  each 
experimental  unit  individually.  Accordingly,  the  analyses  of 
variance  of  Table  V are  appropriate  if  no  covariates  are  used. 

But  we  may  obtain  some  insights  into  the  effects  of  blocking  by 
storms  and  provide  analyses  in  Table  A-IV.  There  were  38  storms, 
some  with  only  one  experimental  unit,  some  with  several  experimen- 
tal units,  all  of  which  were  either  seeded  or  not  seeded,  and 
some  with  both  seeded  or  not-seeded  units.  Storm  effects  were 
totally  or  partially  confounded  with  seeding  effect.  The  analyses 
of  Table  A-IV  were  done  in  such  a way  as  to  consider  the  addi- 
tional effect  of  seeding  after  adjustments  for  storm  effects. 

We  see  that  the  inclusion  of  storm  effects  in  the  model  has  in- 
creased  values  of  R4"  and  reduced  residual  or  error  variances 
(compare  Table  V with  Table  A-IV).  But  the  apparent  effect  of 
seeding  has  disappeared  again.  In  future  similar  experimentation, 
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use  of  storms  for  blocking  should  be  considered,  perhaps  as  sug- 
gested by  the  Statistical  Task  Force,  with  randomization  of  the 
seeding  decision  within  storms.  In  the  Phase  II  Santa  Barbara 
experiment,  a design  change  led  to  seeding  or  not  seeding  all 
convective  bands  within  a storm  because  pf  concerns  for  a per- 
sistence effect  of  seeding.  We  plan  to  do  randomization  analyses 
in  confirmation  of  indications  in  Tables  V and  A-IV,  consistent 
with  a suggestion  by  Gabriel  elsewhere  in  this  volume  that  ran- 
domization analyses  might  be  reserved  for  the  most  critical  com- 
parisons. 

In  further  exploratory  analyses,  we  considered  as  additional 
sources  of  variation  position  of  the  band  within  a storm  and  a 
possible  first-order  carry-over  effect  of  seeding  from  a seeded 
band  to  the  following  band  if  in  the  same  storm.  No  real  effects 
for  positions  or  carry-over  were  found. 

Some  remarks  and  recommendations  can  be  made  after  analysis 
of  the  Phase  I Santa  Barbara  experiment.  We  are  in  near  agree- 
ment with  the  conclusion  of  Elliott  and  Brown  (1971):  "Even  when 
those  bands  not  as  receptive  to  seeding  were  included  in  the 
sample,  the  seeded  to  not-seeded  precipitation  increases  were 
greater  than  50%."  The  means  of  Table  II  show  increases  near 
to  50%  and  the  analyses  of  Table  IV  suggest  significance  near 
to  the  0.05  level. 

Improved  experimental  design  is  needed  but  not  easy  to 
achieve.  Use  of  convective  bands  as  experimental  units 
increases  the  number  of  available  units  per  season  but  raises 
other  problems.  Some  improvements  are  needed: 

(i)  Improved  detection  and -determination  of  "seedable" 
bands . 

(ii)  More  uniform  disperscment  of  raingages  over  regions 
of  interest. 
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(iii)  Improved  determination  and  measurement  of  precip- 
itations attributed  to  particular  convective  bands. 

(iv)  Better  determination  and  measurement  of  covariates 
free  from  possible  seeding  effects. 

(v)  Allowance  for  blocking  by  storms  for  further 
control  of  variation. 

The  quotation  above  supports  the  need  for  (i) . Low 
precipitation  bands  in  Figure  4 cause  difficulty  with  trans- 
formations as  seen  in  Figure  5 and  these  may  not  have  been 
good  "seedable"  bands.  More  uniform  dispersion  of  the  rain- 
gages  over  the  target  area  would  be  desirable,  although  it  is 
understood  that  practical  difficulties  in  so  doing  arise--many 
raingages  used  had  established  locations  and  accessability 
is  a factor  also.  Better  dispersion  of  the  raingages  as 
stated  in  (ii)  should  permit  reduction  of  the  number  used. 

It  is  suspected  that  reading  of  a raingage  for  precipitation 
attributed  to  a given  convective  band  is  very  difficult  and 
introduces  considerable  experimental  variability,  he  do  not 
know  how  to  effect  (iii)  and  the  difficulty  is  offset  by  the 
availability  of  more  experimental  units  in  a season  when  they 
are  taken  to  be  convective  bands.  The  persistence  effect  of 
seeding  discussed  in  the  article  is  more  acute  with  use  of 
convective  bands  as  experimental  units. 

The  major  design  change  needed  in  future  experimentation 
involves  the  measurement  of  suitable  covariates,  covariates 
not  subject  to  possible  changes  due  to  seeding  (iv) . It 
would  appear  to  have  been  better  to  have  taken  the  radiosonde 
observations  at  Vandenberg  Air  Force  Base  than  at  Santa  Barbara 
Airport;  they  would  then  have  been  taken  prior  to  seeding  and 
hopefully  unaffected  by  seeding.  Measurement  of  band 
passage  time  at  Vandenberg  rather  than  at  the  seeding  site 
might  have  been  better  also.  The  use  of  a control  area  seemed 
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an  attractive  idea  but  may  not  be  feasible  unless  chosen  to  be 
unaffected  by  seeding.  Further  meteorological  research  may 
identify  air-mass  covariates  more  closely  correlated  with 
precipitation.  Elliott,  in  correspondence,  has  suggested 
reasons  on  technical  grounds  for  the  use  of  non-linear  functions 

t 

of  the  available  covariates  and  that  very,  careful  formulation 
of  covariates  is  necessary  if  they  are  to  be  effective.  While 
we  respect  his  theoretical  insight,  models  used  may  be  regarded 
as  first-order  approximations  to  more  complex  ones.  Our  use 
of  covariates  reduced  experimental  errors;  the  flaw  is  that 
they  seem  likely  to  have  been  affected  by  seeding.  Blocking 
by  storms  or  future  experiments  seems  feasible  as  suggested 
in  (v)  and  likely  to  be  helpful,  particularly  if  blocks  of 
unequal  sizes  are  used  as  suggested  by  Bril linger,  Tukey  and 
Jones . 

On  the  statistical  side,  transformation  of  the  data  to 
stabilize  variances  and  to  improve  normality  seems  necessary. 
Further  investigation  may  lead  to  better  transforms.  Multi- 
variate methods  similar  to  the  one  used  in  Section  5 do  not 
seem  helpful  and  place  too  stringent  requirements  on  the 
experimenter.  It  remains  to  be  seen  if  use  of  principal 
components,  as  considered  by  Scott  (1978),  will  be  helpful. 

In  spite  of  the  problems  encountered,  we  believe  that 
covariance-regression  analyses,  like  those  of  Section  4, 
give  the  most  potential  for  improved  analysis  of  future 
experiments.  It  will  be  necessary  to  obtain  good  covariates, 
unaffected  by  treatment.  We  do  not  like  the  array  of  univariate 
tests  used  in  Section  2 because  significance  is  difficult  to 
determine.  Confirmation  of  promising  analyses  by  randomization 
tests  may  be  desirable. 
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SUPPLEMENTARY  APPENDIX 


Appendix  Table  A-III  of  the  article  shows  the  regression  coefficients 
for  Model  (5)  of  (4.5)  for  the  transformed  data  for  the  five  target  areas. 
Tables  SA-I  to  SA-IV  of  this  Supplementary  Appendix  below  give  the  regres- 
sion coefficients  for  Models  (6)  to  (8)  of  (4.5). 

Blocking  of  convective  bands  (experimental  units)  by  storms  is  dis- 
cussed in  Section  6 of  the  article  and  analyses  of  variance  are  given 
in  Table  A- IV.  The  model  used  is 


yia’  " * Zi„*  Sti  * fiio’  » ’ 1 V 1 ' 1 S' 


(SA1) 


where  y^  is  the  mean  of  the  transformed  precipitation  responses  for  band 


a of  storm  i for  a specified  target  area,  Zia  = 1 or  0 as  band  a of  storm  i 


was  or  was  not  seeded,  St^  is  the  effect  of  storm  i,  y is  a general  mean, 
and  is  the  residual  error  for  band  a of  storm  i,  taken  in  the  parametric 


model  to  have  variance  a /n^,  where  n^  is  the  number  of  rain gage  observa- 


tions included  in  the  computation  of  y^.  For  some  storms,  nu  = 1 and  there 


is  confounding  between  the  storm  effect  and  the  seeding  variable,  as  also 

occurs  when  all  values  of  Z.  are  either  one  or  zero  for  a storm.  The  ana 

ia 

sis  of  variance  for  a target  area  was  done  through  minimization  of 


E £ n.  (y.  -u-Z.  -St.)' 
l a iov,ia  ia  i' 


(SA2) 


under  a linear  constraint  on  storm  effect  parameters  St^  for  determinancy. 

In  Table  A- IV,  the  sums  of  squares  shown  for  seeding  were  computed  as  the 

additional  reduction  in  the  sum  of  squares  (SA2)  obtained  through  inclusion 

of  Z.  in  the  model. 
ia 

Reference  is  made  also  in  Section  6 of  the  article  to  consideration  of 
position  of  a band  within  a storm  and  a possible  first-order  carry-over 
effect  of  seeding  from  a seeded  band  to  the  following  band  if  in  the  same 
storm.  Since  bands  are  not  of  equal  duration  nor  equally  spaced  within  a 
storm,  the  modelling  is  necessarily  somewhat  crude.  The  full  model  repre- 
sentation, extending  model  (SA1)  above,  has 
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v 

= p+Z.  +St.  +6.  P.  + y.  L+c.  , 
ia  1 ia  k ia  ia 


(SA3) 


where  the  new  parameters  are  defined  as  follows:  6^  = 1 or  0 as  band  a 
of  storm  i is  or  is  not  in  position  k of  the  sequence  of  bands  within  the 
storm,  P^  is  the  effect  of  position  k,  y^  = 1 or  0 as  there  does  or  does 
not  exist  a seeded  band  immediately  preceding  band  a in  the  same  storm  i, 
and  L is  the  carry-over  effect  of  seeding  from  the  preceding  band.  Various 
analyses  for  Target  Areas  (i)  to  (iv)  were  done  with  Model  (SA3)  complete  or 
with  some  terms  omitted;  Target  Area  (v)  was  not  included  in  this  work. 

These  analyses  are  given  in  Tables  SA-V  to  SA-IX.  For  a given  analysis  of 
variance  model  terms  were  included  in  a sequence  ascending  in  the  list  of 
sources  of  variation,  that  is,  seeding  was  always  added  last  to  obtain  the 
additional  reduction  in  the  sum  of  squares  due  to  seeding.  Note  that  the 
effects  of  positions  and  carry-over  from  preceding  seeding  were  negligible 
in  all  analysis  of  variance  tables. 
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Significant  at  level  0.05.  ‘‘Significant  at  level  0.01. 


TABLE  SA-II 


Significant  at  level  0.05.  **Signif icant  at  level  0.01. 


♦Significant  at  level  O.OS.  **Significant  at  level  0.01. 


TABLE  SA-IV 

Regression  Coefficients  for  Model  (9)  for  the  Various 
Target  Areas,  Transformed  Data 
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‘Significant  at  level  0.05.  ‘‘Significant  at  level  0.01. 
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as  are  the  utilities  of  various  statistical  methods.  The  experiment  is  des- 
cribed. The  original  analysis  of  North  American  Weather  Consultants,  who  con- 


ducted the  experiment,  is  reviewed.  Data  summarization  is  reported.  A major 
approach  to  analysis  is  through  the  use  of  cloud-physics  covariates  in  regres- 
sion analyses.  Finally,  a multivariate  analysis  is  discussed.  It  appears 
that  the  covariates  may  have  been  affected  by  treatment  (cloud  seeding)  and 
that  their  use  is  invalid,  not  only  reducing  error  variances  but  removing 
treatment  effect.  Some  recommendations  for  improved  design  of  similar  future 
experiments  are  given  in  a concluding  section,  including  preliminary  trial 
use  of  blocking  by  storms. 


